This fixes issue #45

Additional explanation

Ok, I did a deep dive into the problem, and I found the issue. AzureML compute instance set by default the MSI_ENDPOINT and MSI_SECRET environment variables for on compute instances. Even if this managed identity has no rights. The problem is that the DefaultAzureCredential will prioritize managed identities over CLI logins. This would be fine if DefaultAzureCredential would validate if it selected strategy would work. Instead, it greedily picks the first one that could work, and if it does not work, it does not try the others. So, in this case, it finds the MSI_ENDPOINT env variable, so it decides to go for the AzureMLCredential when this one fails to obtain to get the token, it no longer checks the other options. So in this code:

 try:
        credential = DefaultAzureCredential()
        # Check if given credential can get token successfully.
        credential.get_token("https://management.azure.com/.default")
    except Exception:
        # Fall back to InteractiveBrowserCredential in case DefaultAzureCredential not work
        credential = InteractiveBrowserCredential()

We will always fall back to the InteractiveBrowserCredential.
Sadly, the InteractiveBrowserCredential also does not work since azure compute instances are headless.
We could replace InteractiveBrowserCredential with DeviceCodeCredential. This works, but an annoying side effect is that we needed re-login every time we submit a job.

So, I believe the best course of account would be to check if you are on an AzureML compute instance, and if this is the case, tell DefaultAzureCredential to ignore the managed identity credential. (ManagedIdentityCredential does this by checking for the presence of the MSI_ENDPOINT environment variable`).